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Progress 


This  is  the  first  of  a series  of  reports  of  an  investigation  of  servo 
mechanism  synthesis  under  Navy  Contracts  N7ori30306  and  N?ori30308.  The 
investigation  was  initiated  originally  for  the  purpose  of  studying  the  appli- 
cation of  magnetic  amplifiers  in  servomechanism  design.  It  rapidly  became 
apparent,  however,  that  (A)  it  has  already  been  demonstrated  that  magnetic 
amplifiers  are  applicable  to  servomechanism  problems  in  a wide  variety  of  cases 

(B)  that  such  applications  are  commonly  made  on  a rule-of-thumb  basis  and  that, 

(C)  a more  refined  application  of  magnetic  amplifiers  in  servomechanism  design 
cannot  be  separated  from  the  general  problems  of  servomechanism  synthesis  and 
that  this  project  should  start  with  the  redefinition  of  the  basic  problem, 
that  of  an  optimum  criterion  for  synthesis  on  a realistic  basis. 

In  the  classical  approach  to  servomechanism  analysis  it  is  assumed 
that  (1)  signals  are  of  regular  functional  form,  typically  3tep  functions, 

(2)  such  artificial  criteria  as  phase  margin  or  peak  overshoot  values  are 
satisfactory  as  measures  of  synthesis  success.  More  recently  servomechanism 
approaches  have  been  directed  to  (l)  statistical  properties  of  signals  and, 

(2)  realistic  criteria  both  linear  and  nonlinear.  This  is  the  direction  under- 
taken in  this  particular  study. 

The  results  of  this  study  to  date  are  as  follows: 
a)  A method  of  evaluating  a linear  servomechanism  system  in  the  light 
of  any  type  of  signals  that  can  be  statistically  described,  and  any  type  of 
error  criteria  for  which  a power  series  approximation  can  be  written  (with  the 
common  quadratic  norm  as  one  term  of  the  series).  This  work  is  described 
herein  as  report  number  one. 


b)  A method  of  prescribing  the  optimum  transfer  function  for  systems 
having  certain  specified  time-varying  characteristics,  or  for  signals  having 
certain  specified  time- varying  properties  (including  random  variations),  as 
well  as  the  best  adjustment  for  first  and  second  order  systems  for  a set  of 
commonly  assumed,  statistically  described  signals  which  also  may  have  time 
variations.  In  some  respects  this  work  is  an  extension  of  Wiener's  methods. 
This  work  is  described  in  a report  which  will  be  number  two  of  the  series. 

c)  A method  of  deriving  a transfer  function  for  a magnetic  or  a 
dielectric  amplifier  starting  with  the  B-H  or  D-E  characteristics  of  the  non- 
linear material.  The  method  permits  determining  in  addition  to  the  linear 
terms  the  harmonic  distortion  terms  of  both  the  envelope  and  the  carrier 
functions.  This  material  is  being  readied  for  report  in  the  near  future. 

d)  A review  of  some  classical  and  graphical  methods  of  treating  non- 
linear systems  which  have  been  compiled  but  not  yet  prepared  in  report  form. 

The  work  contained  herein  was  presented  in  preliminary  form  at  the: 
I.R.E.  1953  National  Convention,  "Generalized  Servomechanism  Evaluation", 
Kaufman  and  Caywood,  Session  43:  Remote  Control  Systems. 

Symposium  on  Statistical  Methods  in  Communication  Engineering,  Berkeley  Campus, 
University  of  California,  Aug.  17-18,  1953,  "Generalized  Servomechanism 
Evaluation",  Caywood,  Lyman,  and  Kaufman. 
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The  first  analyses  made  of  servomechanism  systems,  the  present  day 
classical  methods,  allow  first  for  the  calculation  of  absolute  system  stability, 
and  second  for  the  calculation  of  system  response  to  an  input  signal  of  known, 
geometrical  shape.  The  calculation  of  performance  usually  comprises  determining 
the  amount  of  the  first  overshoot  and  the  free  resonant  frequency  in  response 
to  a Heaviside  step  function,  or  the  steady-state  amplitude  and  phase  response 
to  a sinusoidal  signal.  In  instances  when  the  system  is  of  complexity  greater 
than  the  most  simple,  the  response  calculated  may  be  simply  by  rule-of-thumb 
inference  from  the  Heaviside  response  of  the  close  loop  system,  obtained  from 
the  sinusoidal  relation  which  describes  the  system  with  its  feedback  path 
disconnected. 

As  a means  of  evaluating  a servomechanism  the  classical  analysis 
appears  to  have  rather  severe  limitations  in  accuracy  of  a fundamental  origin. 
First  there  can  be  no  prior  knowledge  of  the  exact  input  signal  that  will  be 
demanded  of  the  servomechanism  if  it  is  to  serve  as  a control  device,  rather 
than  a mere  generator  of  waveforms.  Instead,  there  must  be  a certain  repertory 
of  possible  messages,  and  the  occurrences  of  the  individual  possibilities  can 
be  stated  only  in  terms  of  probabilities.  Additional  uncertainty  in  the  function- 
al expressions  for  the  input  signal  result  from  the  inevitable  contamination  of 
the  desired  signal  with  an  undesired  one  (i.e.  noise).  Such  an  undesired  signal 
comprises  another  repertory  of  possible  values,  the  occurrence  of  which  is  in 
part,  if  not  wholly,  unrelated  to  the  occurrence  of  specific,  possible  messages 
of  the  signal  and  again  it  can  be  anticipated  only  in  terms  of  probabilities. 

The  response  to  the  combination  of  signal  and  noise  generally  is  the  one  of 
importance  in  performance  evaluation  or  synthesis  problems. 

A second  limitation  on  the  applicability  of  the  classical  approach  is 
imposed  by  the  difficulty  of  interpreting  the  calculated  system  response  in 
terms  of  a realistic  criterion  of  error-weighting  function.  As  an  example 
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consider  the  case  of  a second  order  system  whose  adjustment  is  limited  to  the 
damping  factor  alone.  Consider  further  the  rather  artificial  condition  of  a 
signal  which  is  a step  function  of  known  magnitude,  and  that  the  performance 
desired  in  the  specific  application  is  such  that  the  error  is  judged  to  be  either 
satisfactorily  small  and  is  called  zero,  or  is  too  large  and  is  called  unity. 

The  integrated  error  due  to  each  input  step  is  shown  in  Figure  1.  Obviously 

the  best  adjustment  depends  on  both  the  permissible  error  as  well  as  on  the  size 
of  the  input  step;  that  is  to  say  it  depends  on  the  application  of  the  system, 
as  well  as  on  the  detailed  characteristics  of  the  signal.  The  illustration 
just  described  is  extended  one  step  further  in  Figure  2.  Here  it  is  assumed 
that  the  input  steps  are  of  various  sizes  and  such  that  they  are  statistically 
described  by  a gaussian  distribution  of  standard  deviation  sigma  and  having  a 
mean  value  of  zero.  The  best  adjustment  is  calculated  to  be  as  shown;  again 

it  is  a function  of  the  input  signal  and  of  the  application.  These  examples 

are  intended  only  to  illustrate  this  dependence  rather  than  to  have  value  in 
themselves. 

On  the  problem  of  a statistical  evaluation  of  servomechanism  systems, 
recognizing  the  necessity  of  a realistic  criterion  of  integrated  error,  the  pro- 
gress to  date  made  by  the  efforts  of  the  authors  has  resulted  in  three  separate 
approaches,  two  described  herein. 

(a)  A computer  which  includes  a system  analogue  and  an  error  criterion 
and  integrating  circuit,  and  which  accepts  statistically  typical  signals  from 

a signal  storage  or  generating  device.  There  results  a single  number  describing 
the  performance  of  the  system. 

(b)  An  analytical  method,  applicable  only  to  linear  systems,  which  makes 
use  of  the  autocorrelation  function  of  the  signal  and  noise  inputs,  and  also  of 
the  higher  moment  correlation  functions  of  the  two  inputs. 
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A . The  Computer 

The  solution  selected  involves  a straight-forward  approach  to  the 
general  problem  utilizing  a model  of  the  system  under  study  to  which  signals  of 
the  correct  statistical  characteristics  are  supplied.  An  evaluation  of  the  per- 
formance of  the  model  is  facilitated  by  the  use  of  a network  which  evaluates 
an  instantaneous  error,  and  which  integrates  a weighted  error  to  give  a single 
number  representative  of  performance.  A block  diagram  of  such  a computer  is 
shown  in  Figure  3* 

The  form  of  the  computer  constructed  at  Carnegie  is  rudimentary. 

The  elements  are  signal  sources,  summing  amplifiers,  a frequency-response  net- 
work comprising  resistors,  condensers  and  cathode  follower  stages  for  isolation, 
a criterion-simulating  network  of  limited  flexibility  (but  with  provisions  for 
external  circuit  connections)  and  an  error-integrating  circuit  with  an  indi- 
cating meter  that  reads  the  average  weighted  error.  The  order  of  the  system 
that  can  be  analyzed  is  virtually  unlimited;  however,  the  types  of  nonlinearities 
permitted  are  quite  restricted  unless  the  electronic  analog  is  replaced  by  a 
more  realistic  model  or  actual  hardware. 

The  computer,  as  described,  is  believed  to  have  considerable  academic 
value  in  addition  to  its  practical  application.  There  are  also  two  extensions 
of  its  present  form  that  suggest  themselves.  One  of  them  results  from  the 
realization  that  a realistic  error  criterion  may  dictate  the  inclusion  of  the 
duration  of  error,  as  well  as  its  instantaneous  amplitude,  in  order  to  establish 
a realistic  measure  of  the  expected  performance  under  use.  The  other  extension 
embodies  the  possibility  of  a controlling  servomechanism  to  repeatedly  readjust 
and  test  the  servomechanism  analogue  and  to  either  record  the  relative  value  of 
performance,  including  its  minimum,  or  to  actually  seek  the  minimum  weighted 
error  as  an  answer  to  the  proper  system  adjustment.  No  progress  has  been  made 
on  these  extensions  of  the  operation  of  the  described  computer. 


\ 


- 4 - 


B.  Utilizing  Correlation  Functions 
An  analytical  method  has  been  developed  for  solving  some  of  the 
problems  that  can  be  treated  by  the  computer  described.  Certain  restrictions 
must  be  tolerated,  namely: 

1.  That  the  servomechanism  system  perform  linear  operations  inveriant 
with  respect  to  the  choice  of  an  origin  in  time. 

2.  That  the  convolution  integral  given  by  the  expression: 

o O 

\ a.  (T)  hi  l-t-T)  JT 

-oO  -J 

possesses  mean  square  or  mean  " " integrability  at  least,  when  A-fr) 

and  H(t-T)  and  mean  " » ^ " integrability  are  as  defined  below. 

This  portion  of  this  paper  is  divided  into  two  parts.  The  first 
comprises  a review  of  the  use  of  correlation  methods  to  evaluate  the  performance 
of  servomechanism  systems  for  cases  in  which  the  input  signal  is  treated  statisti- 
cally. The  second  is  an  extension  of  the  applicability  of  the  methods  reviewed 
in  the  first  part  and  which  is  effected  by  considering  numerical  measures  of 
error,  or  norms,  that  are  more  general  than  the  quadratic.  That  is  to  say,  the 
error  that  may  exist  between  the  output  and  the  input  at  any  instant  is  weighted 
according  to  an  appropriate  weighting  function  that  is  tailored  the  system 
application  under  consideration.  For  input  signals  that  are  statistically 
typical,  the  integrated  (or  average)  value  of  the  weighted  error  is  calculated 
to  form  a single  number  that  is  a measure  of  the  '•poorness"  of  performance.  A 
synthesis  of  the  best  system  for  any  particular  application  is  a matter  of 
minimizing  the  mean  weighted  error  by  proper  choice  of  the  system  parameters. 
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Error,  as  it  applies  to  communications  and  servomechanism  theory,  is 
the  difference  between  the  actual  output  of  a system  and  the  desired  or  ideal 
output. 

In  symbols, 

e = f0-  fd  (2) 

where  <2  z error;  p z actual  output;  P^  z desired  output. 

If,  for  example,  fc  and  are  voltages  as  functions  of  time,  € 
is  also  a voltage  as  a function  of  time,  and  the  mean  square  value  of  error  is 
a measure  of  power  developed  by  the  error  that  must  be  overcome. 

In  any  linear  system,  the  output  is  the  result  of  some  linear  operation 
pn  the  input. 

In  symbols, 

f.  <*)’  A (*)£■  (t)  (3) 

where  A(t)  is  some  linear  operator;^  (t)  is  the  input.  This  could  also 
include  a time  delay,  and  it  also  can  be  written  as  the  convolution  integral  as 
follows : 

o o 

Aftj/-  U)  --  fa.  (rj/-  (t- -r)  Jr  u) 

-60 

^ (rf  is  the  derivative  of  the  system  response  to  a unit  step  and  upon  integra- 
tion becomes  A(t) /t-  (-t)  • The  desired  output  can  be  written  similarly 

fd  (D  = B (5) 

For  the  B operator  there  is  the  choice  of  desiring  the  output  to  be  identical 
with  the  signal  component  of  the  input,  in  which  case  B is  unity.  On  the  other 
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hand  there  may  be  conditions  such  as  the  desirability  of  limiting  the  power 
rating  of  motors  that  suggest  other  choices  of  the  B operator. 

If  the  actual  input  is  the  sum  of  a signal  input  function  fi  (t)  and 
some  random  noise  function  N(t),  both  resulting  from  a stationary  random  process, 
then  the  output  is 

/e<D‘  a (t)  j/,-  ft;  t a/  kj]  (6) 

The  error  is  the  difference  between  the  desired  and  actual  output,  and  can  be 
expressed: 


e(t)s  A (-t)f  (t)  + AH)  N(t)  - B (t) /■  (t) 


(7) 


The  quadratic  norm  can  be  further  expanded: 

£ 

ex(t)  - “f[ A CtJ  - B(V]  f (*)  + A (*)  A/  (t/J- 
e*(t)  ^[A  U)  - B(tj]  f (tff+  ^AH)  M (*)}' 


(8) 


(9) 


+ ZA  (H)[AH)  - BHj]  N (t)f  (t) 

Employing  the  convolution  integral,  as  in  equation  4 above,  and  determining  the 
average  value : T 


*■(?,)  ■ t/rjJ/;-  (■*-%) 


fU(rj-b(TJ (10) 

•*oO 

0 O OO 

+J*.  ( X)  h (■*,  ~ WJ  X 

«o  -<c 

+ n (t,tr,)\^irl)-b(rx)Jl/;  (t -rjjr,  drf  j-t 


-o© 
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Assuming  mean  square  integrability  (if  uniform  convergence  is  not  complete)  the 

order  of  integration  may  be  interchanged  and: 

__  oo  T 

& =fj[a-(r/)  -b  -b  Tf; «-?,)/;  (*-%)  chi 

oo  r t n (11) 

-co  **  -T 

+ z jj o.(Tj[^(r,)-h(rx ;j£/ _l  J" \ (* -x) n a -?,)  *1 

-*P  -T  ^ 


The  three  limiting  processes  can  be  seen  to  define  in  equation  11  the  two  auto- 
correlation functions  for  the  signal  and  the  contaminating  noise,  and  the  cross 
correlation  between  signal  and  noise  respectively.  Expressing  the  auto  correla- 
tions as: 


<P«-(x-x);  h.  (x-x) 


and  the  cross  correlation  by:  (t  . ( *2*  ~ ^ ) 

T H ( 

? - fJ'[*-(X)-b(rl)][«.(rM)->rrM)]  (j>iV  (%.-%) 

+ jfa.  (r.)  a.  (rj  (X-?)drxjr,  (12) 


ao 


+zJJ*..(rx) [*.(?)-  bfr,)J  cpN; 


In  many  cases  the  noise  is  independent  of  the  signal  and  the  cross 
correlation  between  the  two,  , is  zero.  In  this  case  the  last  term  of 


<h,chx 


d-t,cHx 


equation  12  is  removed. 
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Determining  the  best  solution  to  the  synthesis  problem  is  done  by- 
finding  a stationary  value  of  mean  squared  error  which  represents  a minimum.  It 
is  treated  in  the  literature  by  Wiener,  by  Levinson  and  by  others  and  will  not  be 
pursued  further  here. 

The  More  General  Criterion  of  Error 
The  quadratic  norm  has  a certain  realism  in  some  applications,  such  as 
when  the  power  or  energy  of  the  error  is  important.  However,  its  mathematical 
convenience  may  not  be  supported  by  a very  useful  applicability  in  many  situations. 
Consider  fire  control  systems  as  an  example.  Rather  than  an  error  being  "bad" 
in  proportion  to  its  size,  the  target  may  either  be  hit  or  not.  And,  if  it  is 
missed  completely,  it  may  make  no  difference  by  how  much. 

As  a general  means  of  treating  the  problem  of  the  error  criterion,  let 
us  assume  that  whatever  the  realistic  relation  i3,  it  can  be  expanded  as  a power 
series : 


M/ 


Co  t c,e  + ■+  C3  C.  + * 


(13) 


In  cases  of  symmetry  the  odd  terms  are  absent  of  course,  and  in  many  cases  there 
will  be  no  need  to  move  the  origin  by  use  of  Cc  . The  average  value  of  weighted 

(14) 


error : 


e„  - In*  JL  f dt 

zr  J 


-r 

or!  _ r r 

c w - 1 1 m -X  -<  f C0  c/t-  + (c.e<J-t  + / c , s o/~t  + 


W~  I'™  -*=.1  |'-oCiv  1 |c,< 

T-»«o  / J J 

-r  -t 


(15) 


Considering  the  separate  integrals: 

T 

/iV"  _L  j C0  c h - 
T + oo£T  J 

-T 


(16) 
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which  contributes  a shift  in  the  origin.  The  second  terms 

T 

; __L  f c.eft; - c,  e 

T 


(17) 


With  6 given  as  the  difference  of  the  system  operator  A operating  on  message 
plus  noise  and  the  idealistic  operator  B on  message: 

T 

c,e  -- _L Jc,  ^ i/  + a «;  A/<y -b ft;/- cw 

-T 

Applying  the  convolution  integral  as  in  equation  (l)  above: 


T oO  _-i 


C,e  = 

r-*°o  %- 

_r  -oo 

Interchanging  the  orders  of  integration 


oO 


C/  5 ‘ _f[a(Z)-i( rJ]U'^ sffj f-  -V 

-oO  - T 


(20) 


OO  J- 

+ c,  )*(?[&.  -r)dt :jjr 

-oO  T’“  ^ V 


which  is  zero  if  the  overall  average  of  both  ff)  and  N(t)  is  zero.  The  third 
term  represents  the  quadratic  norm  that  is  described  above.  The  fourth  is  zero 
in  cases  of  criteria  that  are  even  functions.  The  next  term: 


Cn  £ " I 1 vr\  L 


'T  4.  (21) 


-aO 
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To  reduce  the  complexity  of  the  expression  consider  cases  in  which  there  is  no 
correlation  between  signal  and  noise.  The  changing  the  order  of  integration  and 
proceeding  as  for  the  quadratic  case  above: 

oO 

c4e"  - <piu<  + 

* fa-  (V-k  (Z) ][*  (rj-b  {%)]  a.1  %)  «.(%)  fcjf 

*.(X)  «-(X)  ■*■(%)  «.(r¥)  <pMI/mJ  JZ,  dr,d% 


where: 

<P„, 0D"  //».  -LCD(t-x)  o(t-x) 

T-+-*0 

-T 

is  a function  that  depends  on  the  third  moment  probabilities, 
and  hence  is  termed  the  third  moment  correlation  function.  It  can  be  calculated, 
or  measured  on  a machine,  in  a manner  analogous  to  the  auto-correlation  functions. 
One  can  proceed  to  the  general  case  which  is  given  by: 
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where : 


(X,  X,  ••••nj 


T 


= i i*  _l  Cv(-t-r,)  D(t-xj  d (*-%)</? 

-r+oo  ZTJ 


-T 


The  correlation  function  yo.~.  o is  again  a higher  moment  one,  in  general  of 
nth  moment  where  n is  also  the  degree  of  the  term  in  the  power  series  being 
considered. 

To  evaluate  the  higher  moment  correlation  functions,  three  possibilities 
are  being  studied  for  convenience  of  application: 

a)  The  direct  approach,  the  construction  of  a machine  that  will  per- 
form the  multiple  multiplications  and  form  the  integral  as  a function 
of  the  various  t*s.  For  reasons  of  compactness  and  simplicity,  mag- 
netic tape  devices  are  considered  as  most  suitable. 

b)  The  possibility  of  simplifying  the  multiplication  process  in  the 
direct  approach  of  a)  by  quantization  of  one  or  more  of  the  members  of 
the  n'th  product  within  the  integral,  using  as  a range  of  variation  of 
the  quantized  number  only  the  plus  or  minus  sign  of  the  function  (i.e. 
infinite  clipping  at  zero  level).  Considerable  work  yet  remains  in 
establishing  regions  of  validity.  The  improvement  in  accuracy  and 
"simplicity  is  expected  to  be  quite  considerable. 

c)  The  digital  computer  approach.  In  many  instances  data  is  already 

of  digital  form,  such  as  when  recorded  on  frames  of  motion  picture  film, 
or  on  punched  cards.  While  relative  merits  and  disadvantages  as  yet 
have  not  been  evaluate4  the  approach  is  quite  straight  forward  and 
merits  serious  consideration. 
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